Predicting Student Outcomes Using Discriminant Function Analysis

Most California community colleges collect copious amounts of data on entering students, most often through the assessment process. However, many times, the data are underutilized: only a few of the data elements captured are used for assessment purposes and the data are not used outside of placement. I have made several attempts to utilize the data, including an attempt to identify variables that would predict success in specific courses using multiple regression. Though this technique can be used to develop models to predict future behavior, it proved to be unfit for helping place students in courses because it can only be used to develop models based on success in a course, not placement into the course. Discriminant function analysis can provide the necessary classification into courses, though the development of a predictive model can prove intimidating. This research explores the limitations of using multiple regression for placement, the use of discriminant function as an alternative, and one method for using discriminant function to provide a model of future behavior. Predicting Student Outcomes RP Group Proceedings 2001 163 Introduction Many research questions in education seek to predict student outcomes based upon a set of independent variables. These variables may include high school information, background information, or scores on a test. Predicting student outcomes is really a process of trying to determine what group an individual student belongs. Should the student be placed into English 1A or a developmental English course? Will the student be more likely to drop out or be put on probation due to poor academic performance during their first semester? Reliable answers to these questions, and others like them, could help colleges tailor services and interventions to target populations and thereby utilize their limited resources more efficiently. The method by which these predictions are made is usually by some statistical technique such as multiple regression. Multiple regression is used in a wide range of applications in social science research (Schroeder, Sjoquist, & Stephan, 1986) and was the initial method of analysis for the research that inspired this paper. However, multiple regression is best used when the outcome, or more generally, the DV, is either dichotomous or interval data (although “with appropriate coding, any comparison can be represented” [Cohen & Cohen, 1983, p. 512]). In the following scenario, I will describe my use of multiple regression, the problem I encountered while created a model, and my ultimate decision to use Discriminant Function Analysis, a decision that ultimately proved the most helpful to the problem at hand. Literature Review College admissions processes often depend on the ability to predict student success. However, the use of a test to help determine admission has traditionally been problematic and continues to be so. Recently, the chancellor of the University of California called for the end of using testing for admissions to college (Selingo & Brainard, 2001). This was not a new call: a plethora of research has shown that standardized tests do not predict success equally well for all groups (Cleary, Humphreys, Kendirick, & Wesman, 1975; Melnick, 1975; Nettles, Thoeny, & Gosman, 1986; Tracey & Sedlacek, 1985) and that standardized tests do not measure what they claim to measure (Riehl, 1994; Sturm & Guinier, 2001). In a recent issue of Boston Review (2001), Susan Sturm and Lani Guiner attack the use of standardized tests in defense of affirmative action, stating: [W]e dispute the notion that merit is identical to performance on standardized tests. Such tests do not fulfill their stated function. They do not reliably identify those applicants who will succeed in college or later in life, nor do they consistently predict those who are most likely to perform well in the jobs they will occupy (p. 4). As an alternative to standardized tests, Strum and Guiner suggest the use of multiple measures as a better way of deciding entry into law school. Often, colleges may rely on two tests as a means of using multiple criteria, but if the two tests are highly correlated with each other, there is needless duplication in measuring the same aspect of a construct (Anastasi, 1982). Because the use of standardized tests has been shown to be problematic, multiple selection methods are being used to predict student success (Ebmeir & Schmulbach, 1989). The use of using multiple measures is called triangulation, the goal of Predicting Student Outcomes RP Group Proceedings 2001 164 which is to “strengthen the validity of the overall findings through congruence and/or complementarity of the results of each method” (Greene & McClintock, 1985, p. 524). This method is used extensively in education for admissions (Markert & Monke, 1990; McNabb, 1990) and involves using a variety of techniques simultaneously to measure a student’s knowledge, skills, and values (Ewell, 1987). Colleges can benefit from combining cognitive and noncognitive variables in predicting student academic success (Young & Sowa, 1992). Because the essence of triangulation is to measure the same construct in independent ways (Greene & McClintock, 1985), the more non-related information gathered, the better the prediction. Triangulation can also minimize or decrease the bias inherent in any particular method by counterbalancing another method and the biases inherent in the other method (Mathison, 1988). For instance, most researchers rely heavily on survey research; however, the assumptions of survey research (e.g., the survey asked all the pertinent questions in a format the respondent can understand) are usually never questioned as a study is designed (Stage & Russell, 1992) which may lead to incomplete or inaccurate conclusions. In the California Community Colleges, the required assessment process dictates the use of multiple measures in placing students into courses. Though the use of a test as one of the multiple measures is highly regulated, the use of multiple measures is not – unless using another test. Because of this, most multiple measures are chosen based on anecdotal or gut reactions and rarely on statistical evidence. It is the lack of research-based decisions for using multiple measures that inspired this research. Collecting Data and Building a Model Many colleges collect more data than they use for analysis on a regular basis. Some examples of data captured from students as a part of assessment include: